REPORT  DOCUMENTATION  PAGE 


Perm 

OMt  No.  0704^tU 


■>_  — mu  itfnfn>n  trf  — -*  **  *  — t“  i  ^owr  wm*  t»»  urn*  tmtwmmq  tmMntetmm.  i— »cwi^  tintwif 

mtfiBruujjiJJn  rmnirimrr"^^**^  «iig  f#wi^no^  tM  toiitwiow  ot  iNform«tieA  comiMV*  «  «i%f  tMtfrifMM  or  mv  otMv  mmci  of 

;i::::^Jjrr^,j^I,Z^^tirt-MiuiMiiioni"ir"rtiin^ir^Ti-fTr  ‘Tr-^n^  -Tpir-^iii  iT^T  *t^-^  ^  —  .  -  ^..  0«»**ro-««  Mv-  WiS;««*« 

Djjgn^rTTff  Um  tlf^  awata twQt»ic«MM^  i  t.  »  a«wu<ti.  ■  *  **r«n;jO|  .«f  .^  iOVOA.  OC  .  41 


yi.  a^PnflT  rrp^  AmO  OAiitJ  CUVfmo 

.^Ant.pmhpr  1QQR  Final  Report:  4/1/94—1/8/98 


S.  PUM04MG  MUMtKIU 


1.  ASUICV  USi  omu  (Los<ifo  btonk)  1 2.  M»0«T  OATl 

I  September  1998 


4.  TITU  AMO  SUSimi 


Nonlinear  and  Structured  Interpolation  for  Robust 
Control 


•.AUTNCMlSI 


Allen  R.  Tannenbaiun 


DAAH04-94-G-0054 


7.  KJtfOMMINa  ORCANUATtON  NAME(S)  AND  AOORISS(U) 

.  Departmoat  of  Electrical  Engineering 
TlnivwcMty  of  Mimieeota 
Minneapolie,  Mmneiota  55455 


9.  SPONSORING/ MONITORING  ACENa  NAME(S)  AND  AOORESS(E5} 

U*  S>  taacf  Beeeareb  Office 
P.  0.  Box  12211 

Beseaxch  Trlesgle  Park,  HC  27709-2211 


1.  RERFORMINO  ORCANIZATION 
RERORT  NUMaSR 


10.  SRONSORING/MONITORMG 
AGENCY  RERORT  NUMBER 


f\K>  %ZM. 


11.  SURRUMENTARV  NOTES 

The  view,  oplnlone  and/oc  fladlngs  eoncalaad  ia  tlua  report  are  tboaa  of  the 
auchor(a)  a^  ehould  sot  be  eonatruad  as  an  official  Dapartacat  of  the  Axmf 
poaltlon,  policy,  or  daclalon.  unleee  ao  designated  by  ether  dectiaentatlon. 

ia«.OISTRIiUTION/AVARaRaiTY  STATEMENT  1 12b.  OISTRIRUTION  CODE 


Approved  for  public  ralaasa;  distribution  unllaltad. 


IS.  ARSTRACT  (AI«Mium200«MrdO  . 

We  have  explored  the  problem  of  e.xtending  the  rme.Tr  theory  to  iiourmear  systems, 
robust  distributed  parameter  control,  and  the  robust  analysis  and  synthesis  of  controllers  for 
systems  in  the  presence  of  various  classes  of  structured  perturbations.  We  have  employed 
operator  theory,  partial  differential  equations,  optimization  theory,  and  invariant  theory  to 
study  the  problem  of  utilizing  visual  information  in  a  feedback  loop  and  the  closely  related 
application  of  visual  tracking.  We  have  considerd  a  combination  of  methods  from  robust 
control  and  computer  vision  for  this  purpose.  Our  approach  to  computer  vision  and  image 
processing  is  based  on  certain  novel  curvature  dependent  evolution  equations  that  may  be 
employed  for  image  enhancement,  denoising,  active  contours,  edge  detection,  morphologj', 
shape  recognition,  shape-from-shading,  stereo  disparity,  and  optical  flow.  We  have  combined 
this  with  our  longstanding  work  in  robust  control,  and  applying  these  these  ideas  to  certain 
benchmark  problems  in  visual  tracking. 


IS.  NUMRER  OF  RAGES 


li.  RRKE  CODE 


14.  SURIECT  HRMS 

Structured  uncertainty,  interpolation  theory,  nonlinear  robust  control, 
distributed  IT**,  Hamilton- Jacobi  equations,  estimation  of  motion. 

•  _ ■ _ _ _ 


iV  SECURE  OASSIFKATliM  I  IS.  SECURITY  CLASSIFICATION  lit.  SECURITY  CLASSIFICATION  20.  UMITATIOM  OF  ASSTRAO 
OR  RERORT  <»  ASSTRAa 

— _ _ _  TTum  AecTgTPn  WarfAfi<tTTTVl%  HL 


Reproduced  From 
lest  Available  Copy 


Final  Report  for  the  Army  Research  Office  Contract 
DAAH04-94-G-0054  entitled 

“Nonlinear  and  Structured  Interpolation  for  Robust  Control” 


Allen  Tannenbaum 
Department  of  Electrical  Engineering 
University  of  Minnesota 
Minneapolis,  MN  55455 
Tel:  612-625-6395 
Email:  tannenba@ece.umn.edu 

September  1998 


Approved  For  Public  Release; 
Distribution  Unlimited. 


19990 


Table  of  Contents 


Contents 


1  Introduction  1 

2  Structured  Singular  Values  and  )U-Synthesis  3 

2.1  Preliminary  Results  on  Structured  Singular  Values .  3 

2.2  Ampliations  of  Perturbations .  4 

2.3  Conditions  for  fi  =  p, .  4 

2.4  Toeplitz  Operators .  6 

3  Causality  and  Robust  Nonlinear  Control  6 

3.1  Causality  in  Nonlinear  Systems .  7 

3.2  Causal  Analytic  Mappings .  9 

3.3  Iterated  Nonlinear  Optimization  .  10 

3.4  Game  Theory  and  Nonlinear  Optimization .  11 

4  Curve  Evolution  in  Controlled  Active  Vision  13 

4.1  Introduction  to  Curve  Evolution  .  . .  13 

4.2  AflBne  invariant  scale-space  .  15 

4.3  Visual  Tracking . ; .  16 

4.4  Geometric  Active  Contours .  16 

5  Bibliography  19 

6  Students  of  Allen  Tannenbaum  Supported  by  DAAH04-94-G-0054  26 


7  Papers  of  Allen  Tannenbaum  and  Collaborators  under  D  AAH04-94-G-0054  27 

1  Introduction 

In  the  research  program  supported  by  DAAH04-94-G-0054,  we  have  carried  out  an  extensive 
study  of  robust  system  control  using  methodologies  from  interpolation  theory,  dilation  theory, 
and  functional  analysis.  Some  of  these  methodologies  have  been  applied  to  certain  benchmark 
problems  involving  visual  tracking. 

A  key  part  of  our  effort  has  centered  around  a  novel  iterative  commutant  lifting  theorem 
which  gives  an  explicit  design  procedure  for  nonlinear  systems  and  captures  the  77°°-control 
problem  in  the  nonlinear  framework  [61],  [62],  [63].  In  this  area,  we  have  defined  a  notion 
of  rationality  for  nonlinear  systems,  and  we  have  proven  that  the  iterative  commutant  lifting 
procedure  produces  rational  controllers  (in  this  nonlinear  sense)  if  we  start  from  rational  data. 
The  procedure  has  already  been  applied  to  certain  systems  with  noninvertible  nonlinearities 
(in  collaboration  with  colleagues  at  Honeywell  in  Minneapolis).  This  framework  has  also  led 
to  new  directions  in  introducing  notions  of  causality  [64]  in  commutant  lifting  theory,  and 
moreover  has  led  to  the  possible  formulation  of  a  global  nonlinear  commutant  lifting  theorem 
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as  a  saddle-point  result.  We  are  particularly  interested  in  the  treatment  (and  understanding 
the  control  limitations)  of  ‘^hard”  nonlinearities  such  as  dead-zone,  backlash,  and  saturation. 

We  have  been  working  as  well  in  distributed  parameter  control  as  part  of  the  research 
sponsored  by  ARO  Grant  DAAH04-94-G-0054.  Recently  a  monograph  [54]  which  extensively 
describes  our  skew  Toeplitz  approach  to  distributed  design  and  analysis  has  been  pub¬ 
lished.  We  have  also  explored  problems  in  //-synthesis  and  analysis;  see  Section  2.  We  have 
developed  a  general  lifting  procedure,  by  which  we  can  interpret  the  D-scaled  upper  bound  for 
the  structured  singular  value  as  a  structured  singular  value  on  a  certain  extended  space.  We 
have  shown  in  which  cases  lifting  is  unnecessary,  i.e.,  the  upper  bound  gives  a  non-conservative 
measure  of  robustness.  This  is  very  important  since  the  upper  bound  for  p  is  log-convex  in 
the  scalings  and  so  can  be  computed,  while  p  cannot.  Our  results  work  directly  for  systems, 
not  just  finite  matrices.  This  allows  us  to  study  broad  classes  of  structured  perturbations 
using  this  tool.  We  have  also  been  continuing  our  work  for  a  rigorous  //-synthesis  procedure 
based  on  structured  interpolation. 

We  have  applied  our  methods  to  the  key  area  of  visual  tracking  which  may  be  employed 
for  a  number  of  problems  in  robotics,  manufacturing,  as  well  as  automatic  target  recognition. 
Even  though  tracking  in  the  presence  of  a  disturbance  is  a  classical  control  issue,  because 
of  the  highly  uncertain  nature  of  the  disturbance,  this  type  of  problem  is  very  difficult  and 
challenging.  Visual  tracking  differs  from  standard  tracking  problems  in  that  the  feedback 
signal  is  measured  using  imaging  sensors.  In  particular,  it  has  to  be  extracted  via  computer 
vision  and  image  processing  algorithms  and  interpreted  by  a  reasoning  algorithm  before  being 
used  in  the  control  loop.  Furthermore,  the  response  speed  is  a  critical  aspect.  As  alluded  to 
above,  we  have  developed  robust  control  algorithms  valid  for  general  classes  of  distributed 
parameter  and  nonlinear  systems  based  on  interpolation  and  operator  theoretic  methods.  In 
our  Army  Research  Office  sponsored  research  project,  we  have  been  explicitly  combining  our 
robust  control  techniques  with  our  novel  approach  to  image  processing  in  order  to  develop 
‘‘state  of  the  art”  visual  tracking  algorithms. 

Indeed,  because  of  our  interest  in  controlled  active  vision,  we  have  been  conducting  re¬ 
search  into  advanced  algorithms  in  image  processing  and  computer  vision  for  a  variety  of  uses: 
image  smoothing  and  enhancement,  image  segmentation,  morphology,  denoising  algorithms, 
shape  recognition,  edge  detection,  optical  flow,  shape-from-shading,  and  deformable  contours 
(“snakes”).  Our  techniques  have  already  been  applied  to  medical  imaging,  and  have  been 
used  to  define  a  novel  affine  invariant  scale-space.  Our  ideas  are  motivated  by  certain  types 
of  geometric  invariant  flows  rooted  in  the  mathematical  theory  of  curve  and  surface  evolu¬ 
tion.  There  are  now  available  powerful  numerical  algorithms  based  on  Hamilton- Jacobi  type 
equations  and  the  associated  theory  of  viscosity  solutions  for  the  computer  implementation  of 
this  methodology.  In  addition  to  systems  and  control,  our  work  in  vision  can  also  has  had  an 
impact  on  ATR.  This  is  based  on  the  nonlinear  filters  we  have  developed  for  image  enhance¬ 
ment  that  are  natural  corollaries  of  our  work.  These  filters  can  smooth  while  preserving  such 
key  features  as  edges  and  ridges. 

A  complete  list  of  publications  which  acknowledges  support  of  DAAH04-94-G-0054  is 
included  in  Section  7.  We  now  sketch  the  work  done  on  this  contract. 
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2  Structured  Singulcu*  Values  and  /^-Synthesis 

During  the  past  funding  period,  we  have  investigated  a  number  of  the  key  properties  of 
the  structured  singular  value  which  was  introduced  into  robust  control  by  John  Doyle  and 
Michael  Safonov  ([39],  [116])  to  handle  problems  involving  structured  perturbations  which 
includes  both  H°°  and  the  multivariable  gain  margin  as  special  cases.  In  particular,  we  have 
introduced  a  new  lifting  or  ampliation  method  for  the  study  of  robustness  under  a  variety  of 
structured  perturbation  models.  We  will  now  briefly  elucidate  some  of  the  key  results  in  this 
area.  Details  about  this  methodology  can  be  found  in  our  papers  [20,  18,  19,  26,  21]. 

2.1  Preliminary  Results  on  Structured  Singular  Values 

In  this  section,  we  will  introduce  some  notation  and  elementary  results  about  the  structured 
singular  value  which  we  will  need  to  explain  our  ampliation  method. 

Let  A  be  a  linear  operator  on  a  Hilbert  space  5,  and  let  A  be  an  algebra  of  operators  on 
S.  The  structured  singular  value  of  A  (relative  to  A)  is  the  number 

Pa{A)  =  l/inf{||X||  :  V  G  A,  -1  e  (t{AX))  . 

(This  quantity  was  defined  in  [39,  116]  under  a  more  restrictive  context.)  Recall  that  in 
system  analysis,  the  structured  singular  value  gives  a  measure  of  robust  stability  with  respect 
to  certain  perturbation  measures.  Unfortunately,  Pa{A)  is  very  difficult  to  calculate,  and  in 
practice  an  upper  bound  for  it  is  used.  This  upper  bound  is  defined  by 

i1a{A)  :=  inf{||J^AX“^||  :  X  G  A', X  invertible}, 

where  A'  is  the  commutant  of  the  algebra  A. 

In  [24,  21],  we  formulated  a  lifting  technique  for  the  study  of  the  structured  singular 
value.  The  basic  idea  is  that  be  shown  to  be  equal  to  the  structured  singular 

value  of  an  operator  on  a  bigger  Hilbert  space.  (In  [24]  this  was  done  for  finite  dimensional 
Hilbert  spaces,  and  then  in  [21]  this  was  extended  to  the  infinite  dimensional  case.)  The 
problem  with  these  results  is  that  the  size  of  the  ampliation  necessary  to  get  /iA(A)  equal 
to  a  structured  singular  value,  was  equal  to  the  dimension  of  the  underlying  Hilbert  space. 
Hence  in  the  infinite  dimensional  case  we  needed  an  infinite  ampliation.  We  should  note,  that 
one  important  result  that  came  out  of  the  work  in  [24]  was  that  for  the  first  time  the  upper 
bound  fl  was  shown  to  be  continuous.  Since  this  is  the  quantity  actually  used  in  fi  analysis 
and  synthesis,  this  result  is  certainly  of  importance. 

In  our  most  recent  work  [20],  we  have  shown  that  in  fact,  one  can  always  get  by  with  a 
finite  ampliation.  (Note  that  in  this  context,  we  will  be  using  the  terms  “ampliation”  and 
“lifting”  interchangeably.)  For  the  block  diagonal  algebras  of  interest  in  robust  control,  the 
ampliation  only  depends  on  the  number  of  blocks  of  the  given  perturbation  structure.  We 
will  moreover,  give  a  new  criterion  below  showing  when  jSA(A)  —  Ma(A),  that  is,  when  no 
lifting  is  necessary  and  so  /2a(A)  gives  a  nonconservative  measure  of  robustness.  This  is  then 
used  to  derive  an  elegant  result  of  Megretski  and  Shamma  [93,  125]  on  Toeplitz  operators. 
See  also  [44,  79,  80]  for  related  work  in  this  area. 

We  will  denote  by  C{S)  the  algebra  of  all  bounded  linear  operators  on  the  (complex, 
separable)  Hilbert  space  S.  Fix  an  operator  A  ^  C{S)  and  a  subalgebra  A  C  C{€).  Observe 
that  A  C  A"  and  A'"  =  (A'')'  =  A'  so  that  we  have  the  inequalities 

//A  (A)  <  /xa"(A),  Ma(A)  =  ^a"(A)  . 


3 


In  our  study  we  will  need  further  singular  values  which  we  now  define.  For  n  €  {1, 2, . .  .,00} 
we  denote  by  the  orthogonal  sum  of  n  copies  of  €,  and  by  the  orthogonal  sum  of  n 
copies  of  T  £  C{€).  Operators  on  5^™)  can  be  represented  as  n  x  n  matrices  of  operators  in 
C{S),  and  T(")  is  represented  by  a  diagonal  matrix,  with  diagonal  entries  equal  to  T. 

Denote  by  An  the  algebra  of  all  operators  on  whose  matrix  entries  belong  to  A,  and 
observe  that  (A„)"  =  (A")„,  and  (A„)'  =  (A')^”)  =  :  T  £  A'}.  Therefore  we  will 

denote  these  algebras  by  A"  and  A^,  respectively. 

2.2  Ampliations  of  Perturbations 

We  can  now  formulate  our  lifting  result  from  [20],  relating  and  JiA{A).  For  finite 

dimensional  S,  a  lifting  result  of  this  type  was  first  proven  in  [24].  The  result  was  then 
generalized  to  the  infinite  dimensional  case  in  [21].  (For  another  proof  of  this  type  of  lifting 
result  in  finite  dimensions,  see  [44].)  In  these  previous  works,  the  lifting  or  ampliation  of 
the  operator  A  and  perturbation  structure  A  depends  on  the  dimension  of  €.  Thus  if  £  is 
infinite  dimensional,  we  get  an  infinite  lifting.  In  Theorem  1  stated  below,  we  only  have  to 
lift  up  to  the  dimension  of  A'  which  in  the  cases  of  interest  in  the  control  applications  of  this 
theory  only  depends  on  the  number  of  blocks  of  the  given  perturbation  structure.  The  proof 
of  the  theorem  makes  use  of  some  work  that  we  did  on  the  relative  numerical  range  and  the 
continuity  of  the  spectrum  on  closed  similarity  orbits  in  [18,  19]  which  we  believe  also  has 
independent  interest. 

Theorem  1  Assume  that  A'  is  a  *-algebra  of  finite  dimension  n.  Then 
for  every  A  £  C{S). 


Remeirk.  In  the  cases  of  interest  in  control, 

A"  =  A, 


and  so  one  has  from  Theorem  1  that 


Ma„(A)  =  Aa(^). 


2.3  Conditions  for  fi  =p> 

We  will  discuss  some  the  conditions  from  [20]  when  =  jl  without  any  need  for  lifting  or 
ampliation.  In  such  cases,  gives  a  nonconservative  measure  of  robustness  relative  to  the 
given  perturbation  structure.  In  the  finite  dimensional  case,  there  have  been  some  results 
of  this  kind,  the  most  famous  of  which  is  that  of  Doyle  [39],  who  showed  that  no  lifting  is 
necessary  for  perturbation  structures  with  three  or  fewer  blocks. 

First  of  all,  call  critical  any  Aq  G  0^t{A)  satisfying 

limsup||(/-€X)Ao(/-eX)-i||  >  \\Ao\l  VX  G  A'. 

«4.o 

Then  we  have 
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Lemma  1  If  Aq  is  a  critical  operator  in  (9^/ (A),  then  it  enjoys  the  following  property  (O): 

0  €  WQiWAofX  -  A*oXAo),  X^A', 

where  Q  =  ||Aolp/  -  AqAq. 

The  next  lemma  is  the  key  step  in  adapting  the  proof  of  Theorem  1  in  order  to  show  that 

mM)  =  ^a(A) 

in  several  interesting  cases. 

Lemma  2  Let  Aq  he  an  operator  on  S  which  satisfies  the  essential  version  of  property  {O), 
property  (0°),  namely 

0  e  -  A*oXAo),  X  €  A', 

where  Q  =  WAqW^I  —  AqAq.  Then  there  exists  a  sequence  C  S,  ||/ifc||  =  1,  A;  =  1, 2, . . 

such  that 

Qhk  0  strongly  and  ((||do|pA’  -  AlXAo)hk,hk)  0, 

for  all  X  e  A'. 

We  can  now  state  another  key  result: 

Theorem  2  If  there  exists  a  critical  operator  Aq  satisfying  property  in  the  closed  A' -orbit 
of  A,  then 

Remark.  Under  the  hypotheses  of  Theorem  2,  when  A"  =  A  (which  happens  in  all  cases  of 
interest  in  control),  we  have  that 

Ma(A)  =  jUA(A). 

Let  L{A'AA')  denote  the  linear  space  generated  by 

A'AA' =  {XAY -.X.Y  £A'}. 

Obviously  L{A'AA')  is  finite  dimensional,  and  therefore  closed.  Hence  Oi^i{A)  C  L{A'AA'). 

Corollary  1  If  for  every  B  G  £(A^AA^),  B  ^  0,  the  norm  of  B  is  not  attained  (that  is, 
there  is  no  h  EH  such  that  ||B/i||  =  ||B||||h||  ^  0^,  then 

PA"  (A)  =  fiA{A). 
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2.4  Toeplitz  Operators 

In  this  section,  we  want  to  use  our  lifting  methodology  in  order  to  derive  a  beautiful  result 
of  Magretski  [93],  and  Shamma  [125]  on  the  structured  singular  value  of  a  Toeplitz  operator, 
i.e.,  a  linear  time  invariant  system. 

Accordingly,  set  S  =  and  let  A  denote  the  multiplication  (analytic  Toeplitz) 

operator  on  S  defined  by 


{Ah){z)  =  A{z)h{z)^  \z\  <1,  h  E  S, 

where 

A{z)  =  [ajk]lk=u  kl<l) 

has  H°°  entries.  Let  A'  be  any  ♦-subalgebra  of  £(C"),  the  elements  of  which  are  regarded 
as  multiplication  operators  on  S.  Note  that  in  this  case.  A"  =  A  is  the  algebra  generated  by 
operators  of  the  form 

{Bh){z)  =  B{z)h{z),  \z\<l,  hES 
with  B(z)X  =  XB(z),  \z\  <  I,  X  E  A'  as  well  as  of  the  form 


hi  1 

■  Yhi  ' 

h2 

_ 

Yh2 

-  . 

1 

S 

1 _ 

with  Y  E  C{H^{C))  arbitrary.  We  can  now  state: 

Lemma  3  Let  Aq  be  an  analytic  Toeplitz  operator.  Then  if  Aq  has  property  {O),  it  also  has 
property  (Cl°). 

Corollary  2  ([93,  125])  For  A  and  A'  as  above,  we  have  that 

PAiA)  =pa{A). 


3  Causality  and  Robust  Nonlinear  Control 

In  our  work  in  nonlinear  robust  control,  and  especially  the  extension  of  H°°  to  the  nonlinear 
framework,  we  have  discovered  a  number  of  intriguing  problems  which  do  not  occur  in  the 
linear  case.  Indeed,  besides  the  theoretical  and  practical  questions  involved  in  finding  an 
implementable  design  methodology,  it  is  interesting  to  note  that  certain  associated  problems 
of  causality  have  arisen  in  working  in  this  area,  which  we  would  like  to  briefiy  sktch.  In  fact, 
as  a  result  of  this  effort  we  have  been  able  to  put  an  explicit  causality  constraint  in  commutant 
lifting  theory  for  the  first  time  [49,  51,  64,  17]. 

There  have  been  several  attempts  to  extend  dilation  theoretic  techniques  to  nonlinear 
input/output  operators,  especially  those  which  admit  a  Volterra  series  expansion  (see  [50] 
and  the  references  therein).  In  several  of  these  approaches,  one  is  reduced  to  applying  the 
classical  (linear)  commutant  lifting  theorem  to  an  Lf^-space  defined  on  some  D”  (where  D 
denotes  the  unit  disc) .  Now  when  one  applies  the  classical  result  to  Z?”  (n  >  2) ,  even  though 
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time-invariance  is  preserved  (that  is,  commutation  with  the  appropriate  shift),  causality  may 
be  lost.  Indeed,  for  analytic  functions  on  the  disc  D,  time-invariance  (that  is,  commutation 
with  the  shift)  implies  causality.  For  analytic  functions  on  the  n— disc  (n  >  1),  this  is  not 
necessarily  the  case.  For  dynamical  system  control  design  and  for  any  physical  application, 
this  is  of  course  a  major  drawback.  Hence  for  a  dilation  result  in  /f^(Z?”)  we  need  to  include 
the  causality  constraint  explicitly  in  the  set-up  of  the  dilation  problem.  We  will  discuss  a  way 
of  doing  this  now  based  on  [64,  50,  51],  and  some  of  the  beautiful  associated  problems  that 
arise  in  operator  theory  and  systems  and  control  as  a  result  of  this  method. 

3.1  Causality  in  Nonlinear  Systems 

Recall  that  causality  basically  means  that  for  a  given  input/output  system  the  past  output 
is  independent  of  the  future  inputs.  This  may  be  given  precise  mathematical  formulation  in 
terms  of  a  family  of  projections  which  we  shall  now  do.  We  follow  treatment  of  [64]. 

Let  S  denote  an  isometry  on  a  Hilbert  space  Q,  and  let  T  denote  a  contraction  on  a 
Hilbert  space  %.  Let  Pjo,  J  >  1  denote  a  sequence  of  orthogonal  projections  in  Q  satisfying 
the  following  conditions: 


-Pio  <  P20<---  .  (1) 

Pjo  <  I-S^S*^  i  =  l,2,...  (2) 

Pj^i,oSiI-Pjo)  =  0  j  =  (3) 

Next  U  K  K  will  denote  a  minimal  isometric  dilation  of  the  given  contraction  T 
defined  on  7i.  Let  3:0-^%  intertwine  S  with  U,  that  is, 

UB  =  BS.  (4) 

Note  that  this  latter  condition  implies  WB  =  BS^  {j  >  1),  hence  U^U*^BS^  S*^  =  BS^B*^, 
and  so 

(/  -  WU*^)B  =  (/  -  UW*^)B{I  -  S^S*^)  i  =  1, 2, . . . .  (5) 

We  now  make  the  following  key  definition: 

Definition.  An  operator  B  satisfying  (4)  is  called  (Pio,  F20, . .  .)-causal  (and  if  the  sequence 
{Fjo}^i  is  fixed,  causal)  if 

(7  -  UW*^)B  =  {I-  WU*^)BPjo  j  >  1,  (6) 

or  equivalently, 

(/  -  Pjo)B*  =  (7  -  Pjo)B*WU*^  j  >  1.  (7) 


Note  that  B  is  always  (7  —  SS*,  7  —  S^S*^, . .  .)-causal.  In  what  follows  the  sequence 
Fio,  F20, .  • .  will  be  fixed  and  causality  will  always  be  defined  relative  to  this  sequence. 

Next  we  let  A  :  ^  ^  be  an  operator  intertwining  S  and  T,  that  is,  AS  =  T A.  Then 

an  intertwining  lifting  (or  dilation)  of  A  is  an  operator  B  :Q  -¥  1C  such  that  BS  =  UB,  and 
PB  =  A  where  P  :  K  denotes  orthogonal  projection. 
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We  define 


Voo{A)  :=  inf{||B||  :  JB  is  a  causal  intertwining  dilation  of  A}, 

and 

Ijl{A)  :=  min{M  >  0  :  ||A||  <  M,  \\{I  -  Pjo)A*h\\  <  M\\T*^h\l  henj>l}. 

We  can  then  show  that 

fi{A)  <  Voc{A).  (8) 

We  will  also  need  a  functional  which  lies  between  //(A)  and  i/oo(A).  To  this  aim,  we  call 
a  sequence  of  operators  Fj  :  Qj  :=  (I  —  Poj)G  —>  Ti,  j  =  0,2, . . .  a.  resolution  of  A  if 

Fo  =  A 

Vj  >  0, 

F,-  =  F,+i5|g,-  Vi>0. 

(Note  that  we  take  Pqo  :=  0,  so  that  Qq  =  G-)  We  now  define 

/2(A)  := 

min{M  >  0  :  |1A||  <  M  and  there  exists  a  resolution  of  A,  Fj,  with  ||Fj||  <  M,  \/j  >  0}. 
Then,  we  have  the  following  results  from  [64]: 

Theorem  3  (Causal  Commutant  Lifting  Theorem)  Notation  as  above.  Then 

i'oo(A)  = /2(A). 


We  also  have  the  following  variant  of  the  Causal  Commutant  Lifting  Theorem: 


Corollary  3  IfkeiT  —  {0},  then  /i(A)  =  Voo{A). 


In  the  case  of  the  compressed  shift,  which  is  of  course  of  greatest  interest  in  control,  based 
on  these  results  we  have  developed  a  completely  constructive  method  for  designing  nonlinear 
causal  controllers  which  generalize  the  H°°  procedure.  Indeed,  in  this  case  one  can  write 
down  explicitly  the  causal  dilations  (see  [49]).  In  fact,  in  [49,  50]  via  a  Reduction  Theorem, 
we  show  how  the  construction  of  a  causal  dilation  may  be  reduced  to  a  classical  interpolation 
problem. 
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3.2  Causal  Analytic  Mappings 

We  will  now  specialize  the  discussion  to  analytic  mappings  on  Hilbert  space.  We  use  the 
standard  definitions  from  [64]. 

So,  we  consider  an  analytic  map  <f>  with  Q  =  %  =  (the  standard  Hardy  space  on  the 
disc  D).  Note  that 

®  .  .  .  0  ^2  ^  ^ 

where  we  map  (2  in  the  i-th  place)  to  2r,-,  i  =  1,  •  •  • ,  n.  In  the  usual  way, 

we  say  that  (t>  shift-invariant  (or  time-invariant)  if 

=  S<l>n  Vn>l, 

where  5  :  denotes  the  canonical  unilateral  right  shift.  (Equivalently,  this  means 

that  S<f>  =  0  o  5  on  some  open  ball  about  the  origin  in  which  <f>  is  defined.) 

Now  set 

Pu)  ®“'®  Pq)  (”  t™®s),  i  >  1,  n  >  1, 

where 

PlJ)  ■■=  I  - 

Then  we  say  that  (j)  is  causal  if 

P(j)^n  —  i  ^  ^  ^  1* 

For  <j) :  ->  linear  and  time-invariant  (i.e.,  intertwines  with  the  shift),  it  is  easy  to  see 

that  (j>  is  causal.  In  the  nonlinear  setting  however,  time-invariance  may  not  imply  causality. 
As  a  concrete  example,  let  (f>o  :  be  a  linear  operator  such  that  S®^(j)o  =  (t>oS, 

defined  by 

00 

(Mf  ®  5)) (^)  :=  ifk+i9k  +  fk9k  +  fkgk+i)z^, 
k=0 

where 

00  00 

fi^)  =  Y,  9iz)  =  Y  9kz:'‘- 

k=Q  k=0 

Now  set 

m-=Mf®f),  f^H\ 

Then  <f>  is  an  analytic  time-invariant  map.  (In  fact  is  a  homogeneous  polynomial  of  degree 
2.)  But  <l>  is  not  causal.  Indeed, 

{P{i)Hf)){^)  =  2/1/0 -h/o^  z€D 
{Pii)HP(i)f)m  =  /o^ 

Thus  P(i)<l>{f)  ^  P(i)<t>{P{i)f)i  for  example  for  /(z)  :=  1  -|-  z  for  z  G  D.  It  is  for  this  reason, 
that  a  causality  constraint  must  be  explicitly  included  for  nonlinear  H°°  design. 

Now  it  is  easy  to  show  that  the  conditions  (1),  (2),  and  (3)  given  above  are  verified  for 

the  pjj^  and  so  the  causal  commutant  lifting  theorem  applies.  In  the  next  section,  we  will 
see  how  one  can  base  an  iterative  design  procedure  on  this  result  [50,  64,  62,  51]. 
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3.3  Iterated  Nonlinear  Optimization 

We  will  now  consider,  how  the  above  ideas,  lead  to  a  an  optimization  procedure  for  nonlinear 
systems  which  generalizes  H°°  in  a  completely  natural  manner.  For  convenience,  we  will 
only  treat  SISO  systems  here.  Let  us  call  an  analytic  input/output  operator  ^  -> 

admissible  if  is  is  causal,  time-invariant,  majorizable,  and  <^(0)  =  0.  Denote  the  set  of 
admissible  operators  by  Ca-  In  what  follows  below,  we  assume  P,W  €Ca,  and  that  W  admits 
an  admissible  inverse. 

We  consider  the  (one  block)  problem  of  finding 

fj,s-=iaf  sup  ||[(7-|- PoC)"^  o  W]w||,  (9) 

^  IH|<5 

where  we  take  the  infimum  over  all  stabilizing  controllers.  (In  what  follows,  we  let  ||  ||  denote 
the  2-norm  ||  ||2  on  H'^  as  well  as  the  associated  operator  norm.  The  context  will  make  the 
meaning  clear.)  Thus  we  are  looking  at  a  worst  case  disturbance  attenuation  problem  where 
the  energy  of  the  signals  v  is  required  to  be  bounded  by  some  pre-specified  level  6,  (In  the 
linear  case  of  course  since  everything  scales,  we  can  always  without  loss  of  generality  take 
^  =  1.  For  nonlinear  systems,  we  must  specify  the  energy  bound  a  priori.)  Then  one  sees 
that  (9)  is  equivalent  to  the  problem  of  finding  the  problem  of  finding 

fis=  inf  sup  \\{W-Poq)v\\.  (10) 

The  iterated  causal  commutant  lifting  procedure  gives  an  approach  for  approximating  a 
solution  to  such  a  problem.  Briefly,  the  idea  is  that  we  write 

W  =  Wi+W2  +  ---, 

P  =  Pi  +  Pz  +  •  •  ■ ) 

g  =  91  +  92  + 

where  Wj,Pj,qj  are  homogeneous  polynomials  of  degree  j.  Notice  that 

ps  =  S  inf  J\Wx-Piqi\\  + 0(6%  (11) 

where  the  latter  norm  is  the  operator  norm  (i.e.,  norm).  From  the  classical  commutant 
lifting  theorem  we  can  find  an  optimal  (linear,  causal,  time-invariant)  qi,opt  €  H°°  such  that 

fis  =  <5||Wi  -  Piqi,opt\\  +  0{S^).  (12) 

Now  the  iterative  procedure  gives  a  way  of  finding  higher  order  corrections  to  this  lin¬ 
earization.  Let  us  illustrate  this  now  with  the  second  order  correction.  Indeed,  having  fixed 
the  linear  part  qi,opt  of  q  in  (10),  we  note  that 

Wiv)  -  P{q{v))  -  {Wi  -  Piqi,opt)iv)  = 

W2(v)  -  P2{qi,opt{v))  -  Pi92(v)  +  higher  order  terms. 

Regarding  W2,  P2, 92  as  linear  operators  on  ®  C)  as  above,  we  see  that 

sup  ||(W  -  P  o  g)(t;)  -  (Wi  -  Piqi,opt)v\\  <  6^\\W2  -  P192II  +  0(6% 

\\v\\<S 
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where  the  ‘^weight”  W2  is  given  by 

W2  :=  W2  —  P2(Qi,opt  ®  9i,opt)- 

Using  the  control  version  of  the  causal  commutant  lifting  theorem  (see  [50]  and  Section  4.1 
below),  we  can  now  construct  an  optimal  admissible  q2,opti  2ind  so  on.  This  is  our  iterated 
optimization  procedure. 

In  short,  instead  of  simply  designing  a  linear  compensator  for  a  linearization  of  the  given 
nonlinear  system,  this  methodology  allows  one  to  explicitly  take  into  account  the  higher  order 
terms  of  the  nonlinear  plant,  and  therefore  increase  the  ball  of  operation  for  the  nonlinear 
controller.  Moreover,  if  the  linear  part  of  the  plant  is  rational,  our  iterative  procedure  may 
be  reduced  to  a  series  of  finite  dimensional  matrix  computations.  (See  [50,  61]  for  discussions 
of  rationality  in  the  nonlinear  framework.) 

In  our  continuing  research,  we  will  be  developing  the  above  framework  into  an  imple- 
mentable  design  procedure.  Accordingly,  we  are  having  a  doctoral  student,  program  part 
of  the  procedure  symbolically  in  Mathematica.  Notice  that  since  the  nonlinear  optimization 
problem  has  now  been  reduced  to  a  series  of  linear  problems,  we  can  use  any  of  the 
standard  methods  in  order  to  iteratively  design  the  nonlinear  compensator.  We  will  then  be 
looking  to  apply  this  to  some  specific  nonlinear  design  examples.  For  some  preliminary  work 
in  this  direction,  see  [49,  50]. 

3.4  Game  Theory  and  Nonlinear  Optimization 

We  have  just  considered  an  iterative  commutant  lifting  approach  to  nonlinear  system  design. 
The  iterative  communtant  lifting  technique  is  basically  a  local  analytic  method  for  nonlinear 
system  synthesis.  We  have  also  been  exploring  a  very  different  approach  applicable  to  certain 
systems  with  saturations  (and  “hard”  noninvertible  nonlinearities)  based  on  a  game-theoretic 
interpretation  of  the  classical  commutant  lifting  theorem  [65].  This  motivates  us  to  formulate 
a  nonlinear  commutant  lifting  result  in  such  a  saddle-point,  game-theoretic  framework. 

A  related  approach  to  nonlinear  design  has  already  been  employed  by  a  number  of  re¬ 
searchers;  see  [10,  11,  12,  13,  76,  77,  78,  135,  14]  and  the  references  therein.  As  is  well  known, 
game  theoretic  ideas  have  already  been  extensively  applied  in  linear  theory. 

In  our  research,  instead  of  considering  general  nonlinear  systems  we  have  limited  ourselves 
to  the  concrete  (but  certainly  interesting  case)  of  linear  systems  with  input  saturations.  Such 
systems  occur,  of  course,  all  the  time  in  “nature.”  We  should  add  that  a  similar  approach 
should  be  valid  for  many  of  the  hard,  memoryless,  noninvertible  nonlinearities  which  appear 
in  control. 

Mathematically,  the  case  of  the  saturation  is  very  interesting  since  it  represents  a  non¬ 
invertible  nonlinearity,  and  in  a  certain  sense  is  a  nonlinear  analogue  of  an  inner  (non¬ 
minimum  phase)  element.  In  fact,  it  seems  that  the  saturation  “acts”  like  an  inner  function 
whose  spectrum  is  spread  throughout  the  unit  circle.  Thus  the  problem  of  sensitivity  mini¬ 
mization  for  such  elements  (that  is,  a  form  of  weighted  inversion)  is  particularly  difficult,  and 
will  be  an  important  topic  to  be  considered  in  our  research  program.  Again,  we  will  discuss 
some  preliminary  results  along  these  lines  based  on  [65]. 

In  order  to  motivate  our  game-theoretic  approach  to  nonlinear  we  will  first  give  a 
“saddle-point”  interpretation  of  the  classical  Sarason  theorem  [129]  in  a  special  case.  We  let 
w^m  £  with  m  inner.  Set  H{m)  :=  we  let  Prnm)  :  >  H{m)  denote 
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orthogonal  projection,  and  S{m)  denote  the  compressed  shift.  We  let  ||  ||  denote  the  2-norm 
II  II2  on  as  well  as  the  associated  induced  operator  norm. 

In  [65],  we  prove  that 


Now  it  is  easy  to  show  there  always  an  optimal  qo‘,  see  e.g.,  [65].  We  now  assume  that 

||«,(£;(m))||.„  <  ||<«(S(m))||, 

where  ||  ||ess  denotes  the  essential  norm.  Then  there  exists  fo  G  H^,  ||/o||  =  1  (a  maximal 
vector),  such  that 

||(u;-m<|ro)(5'(m))/o||  =  l|w(5(m))/o|| 

=  lk(5(m))|| 

=  ||(w>-»^go)(5'(m))||. 

Now 

-  ‘>nqo)fo  =  (w-  mqo)iS{m))fo  =  w{S{m))fo, 

(ty  -  mqo)fo  =  w{S{m))fo. 

So 

\\{w-mqo)f\\  <  \\{w  -  mqo){S{m))fo\\  =  \\iw  -  mqo)fo\\ 
for  all  /  G  ll/ll  <  1.  Moreover, 

||u;(5(m))/o||  =  Wiw  -  mq){S{m))fo\\  <  ||(w  -  mg)/o||. 


Hence,  we  get  that 

\\{w-mqo)f\\  <  \\{w-mqo)fo\\  <  \\{w  -  mq)fo\\  (13) 

for  all  /  G  H‘^,  ||/||  <  1,  and  for  all  q  G  H°°.  It  is  a  nonlinear  analogue  of  the  saddle- 
point  condition  (13)  that  we  want  to  analyze  for  saturated  systems.  Indeed,  assuming  the 
saddle-point  condition  (13),  in  [65]  we  derive  all  of  the  standard  consequences  of  the  Sarason 
theorem.  Thus  it  is  precisely  the  existence  of  a  saddle-point  which  we  would  like  to  explore 
in  the  nonlinear  setting. 

By  virtue  of  interpretation  of  the  commuting  lifting  theorem  as  asserting  the  existence  of 
a  saddle-point,  we  would  like  to  derive  a  global  approach  to  sensitivity  minimization  for  input 
saturated  systems.  Thus  for  ag,  a  saturation  of  magnitude  6  <  1  (see  [65]  for  all  the  precise 
definitions),  and  m  G  H°°  inner,  we  want  to  know  when  there  exist  fo  G  ||/o||  <  Ij 
continuous,  causal,  time-invariant,  such  that 

\\{w  -  mere  o  qo)f\\  <  ||(w  -  mag  o  qo)fo\\  <  IK^  -  rnag  o  q)fo\\ 

for  all  /  G  H^,  ||/1|  <  1,  9  continuous,  causal,  time-invariant.  Such  a  qo  (when  it  exists)  will 
correspond  to  the  optimal  compensator,  and 

fJ,  :=  \\{w  -  mag  o  qf)fo\\ 
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will  be  the  optimal  performance  in  the  weighted  sensitivity  minimization  problem.  But  this 
is  equivalent  to  finding  Qq  =  9o(/o)  G  such  that 

||(«?  -  m<T0  o  qg)f\\  <  \\w{fo)  -  Tnae{go)\\  <  ||(t«  -  rnae  o  q)fo\\.  (14) 

The  approach  we  want  to  explore  now  in  our  present  research  is  to  follow  an  analogous 
line  of  reasoning  which  we  employed  in  our  analysis  of  the  saddle-point  condition  in  the  linear 
case.  We  believe  that  at  the  end  of  the  line,  we  will  discover  a  novel  nonlinear  commutant 
lifting  theorem  valid  on  a  convex  space  which  can  be  used  to  develop  a  global  robust  design 
procedure  for  nonlinear  plants  with  hard  nonlinearities. 


4  Curve  Evolution  in  Controlled  Active  Vision 

In  the  past  few  years,  we  have  become  very  interested  in  visual  tracking  and  the  general  area 
of  the  use  of  visual  information  in  a  feedback  loop.  This  is  a  central  area  in  which  the  robust 
control  methods  developed  over  the  past  fifteen  years  could  have  a  major  impact.  In  order 
to  work  on  visual  tracking,  we  have  had  to  learn  some  of  the  key  techniques  from  image 
processing  and  computer  vision,  which  has  led  in  turn  to  a  new  research  direction.  Indeed, 
we  have  been  using  geometric  invariant  flows  for  various  problems  in  active  vision.  These 
flows  themselves  are  very  much  motivated  by  ideas  in  optimal  control;  see  [87].  We  will  now 
discuss  some  of  the  key  ideas  in  curve  evolution.  These  will  be  applied  to  certain  problems 
in  controlled  active  vision  as  described  below. 


4.1  Introduction  to  Curve  Evolution 


A  geometric  set  or  shape  can  be  defined  by  its  boundary.  In  the  case  of  bounded  planar 
shapes  for  example,  this  boundary  consists  of  closed  planar  curves.  We  will  only  deal  with 
closed  planar  curves,  keeping  in  mind  that  these  curves  are  boundaries  of  planar  shapes. 

A  curve  may  be  regarded  as  a  trajectory  of  a  point  moving  in  the  plane.  Formally,  we 
define  a  curve  C(*)  as  the  map  C{p)  :  (where  denotes  the  unit  circle).  C  can 

be  written  using  Cartesian  coordinates,  i.e.,  C{p)  =  [^(jP)j  y(p)]^j  where  x(-)  and  j/(‘)  are 
maps  from  to  R.  We  assume  that  all  of  our  mappings  are  sufficiently  smooth,  so  that 
all  the  relevant  derivatives  may  be  defined.  We  also  assume  that  our  curves  are  have  no 
self-intersections,  i.e.,  are  embedded. 

We  now  consider  plane  curves  deforming  in  time.  Let  C(p,  t)  :  x  [0,  r)  R^  denote  a 

family  of  closed  embedded  curves,  where  t  parametrizes  the  family,  and  p  parametrizes  each 
curve.  Assume  that  this  family  evolves  according  to  the  following  equation: 


{ 


^  =  c,f  +  0J<t 

C(p,0)  =  Co{p) 


(15) 


where  /v  is  the  inward  Euclidean  unit  normal,  T  is  the  unit  tangent,  and  a  and  /3  are  the 
tangent  and  normal  components  of  the  evolution  velocity  o,  respectively.  In  fact,  it  is  easy 
to  show  that  Img[C(p,  f)]  =  Img[d(u;,t)],  where  C{p,t)  and  C{w,i)  are  the  solutions  of 

Ct  =  df  -1-  pM'  and  Ct  =  PN", 

respectively.  (Here  Img[-]  denotes  the  image  of  the  given  parametrized  curve  in  R^.)  Thus  the 
tangential  component  alfects  only  the  parametrization,  and  not  Img[-]  (which  is  independent 
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of  the  parametrization  by  definition) .  Therefore,  assuming  that  the  normal  component  (5  oiv 
(the  curve  evolution  velocity)  in  (15)  does  not  depend  on  the  curve  parametrization,  we  can 
consider  the  evolution  equation 

(16) 

where  ^  =  u  ■  N,  i.e.,  the  projection  of  the  velocity  vector  on  the  normal  direction. 

The  evolution  (16)  was  studied  by  different  researchers  for  different  functions  /?.  This  type 
of  flow  was  introduced  into  the  theory  of  shape  in  [82,  83,  84,  85].  One  of  the  most  studied 
evolution  equations  is  obtained  for  (3  =  k,  where  k  is  the  Euclidean  curvature  [73,  128]: 


k7 


(17) 


Equation  (17)  has  its  origins  in  physical  phenomena  [3,  72].  It  is  called  the  Euclidean 
shortening  flow^  since  the  Euclidean  perimeter  shrinks  as  fast  as  possible  when  the  curve 
evolves  according  to  (17);  see  [72].  Gage  and  Hamilton  [70]  proved  that  a  planar  embedded 
convex  curve  converges  to  a  round  point  when  evolving  according  to  (17).  (A  round  point  is 
a  point  that,  when  the  curve  is  normalized  in  order  to  enclose  an  area  equal  to  tt,  it  is  equal 
to  the  unit  disk.)  Grayson  [71]  proved  that  a  planar  embedded  non-convex  curve  converges 
to  a  convex  one,  and  from  there  to  a  round  point  from  Gage  and  Hamilton  result.  Note  that 
in  spite  of  the  local  character  of  the  evolution,  global  properties  are  obtained,  which  is  a  very 
interesting  feature  of  this  evolution.  For  other  results  related  to  the  Euclidean  shortening 
flow,  see  [3,  4,  70,  71,  72,  85,  137]. 

Next  note  that  if  s  denotes  the  Euclidean  arc-length,  then  [128] 


d^C 


Therefore,  equation  (17)  can  be  written  as 


Ct  —  Cs 


Equation  (18)  is  not  linear,  since  5  is  a  function  of  time  (the  arc-length  gives  a  time  dependent 
parametrization).  This  equation  is  also  called  the  geometric  heat  equation. 

Another  interesting  example  is  obtained  when  one  sets  (3  =  1  in  equation  (16): 


(19) 


This  equation  simulates,  under  certain  conditions,  the  grassfire  flow  [30,  82,  83,  85,  123]. 
(More  precisely,  the  unique  weak  solution  of  (19)  which  satisfies  the  entropy  condition  [123] 
gives  the  grassfire  flow.)  This  grassfire  flow  is  also  the  morphological  scale-space  created 
by  a  disk.  Moreover,  one  can  prove  that  with  different  selections  of  /3,  other  morphological 
scale-spaces  are  obtained  [87]. 

In  [82,  83,  85],  we  have  studied  the  following  equation  in  order  to  develop  a  hierarchy  of 
shape. 


f  =  (!  +  «)/?. 


(20) 
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If  €  —^  0  in  (20),  the  grassfire  flow  is  obtained,  and  this  introduces  singularities  (shocks) 
in  the  evolving  curve.  (The  shocks  define  the  well-known  skeleton.)  On  the  other  hand, 
if  €  ->  oo,  equation  (20)  reduces  to  the  classical  Euclidean  curve  shortening  flow,  which 
smoothes  the  curve  [71,  124].  The  combination  of  these  two  opposite  features  gives  very 
interesting  properties  [82]. 

When  a  curve  evolves  according  to  (20) ,  the  evolution  of  the  curve  slope  satisfies  a  reaction- 
diffusion  equation  [126].  The  reaction  term,  which  tends  to  create  singularities,  competes  with 
the  diffusion  term  which  tends  to  smooth  the  curve.  For  each  different  value  of  €,  a  scale- 
space  is  obtained  by  looking  at  the  solution  of  (20),  and  considering  the  time  t  as  the  scale 
parameter.  We  have  called  the  set  of  all  the  scale-spaces  obtained  for  all  values  of  t,  the 
reaction-diffusion  scale-space  [83]. 

In  particular,  we  see  that  the  much  studied  Euclidean  shortening  flow  (equation  (17))  de¬ 
fines  an  Euclidean  invariant  scale-space  (the  equation  admits  Euclidean  invariant  solutions) . 
In  contrast  with  other  scale-spaces,  like  the  one  obtained  from  the  classical  linear  heat  equa¬ 
tion,  this  one  is  a  full  geometric  scale-space.  The  progressive  smoothing  given  by  k  (or  Css), 
is  geometrically  intrinsic  to  the  curve. 

We  now  discuss  the  affine  analogue  of  the  Euclidean  shortening  flow.  (The  aflSne  group 
SA2  is  the  group  generated  by  unimodular  transformations  and  translations  of  R^.  Under 
certain  natural  conditions,  it  provides  a  good  approximation  to  the  full  group  of  perspective 
projective  transformations.)  Then  in  [99,  118],  we  show  that  the  simplest  non-trivial  affine 
invariant  flow  in  the  plane  is  given  by 

Ct  =  (21) 

The  question  now  is  what  happens  when  a  non-convex  curve  evolves  according  to  (21). 
The  following  result  answers  this  question  [5]: 

Theorem  4  Let  C(‘,0)  :  5^  6e  a  smooth  embedded  curve  in  the  plane.  Then  there 

exists  a  family  C  :  5^  x  [0,  T)  satisfying 

such  that  C(-,t)  is  smooth  for  all  t  <  T,  and  moreover  there  is  a  to  <  T  such  that  for  all 
t  >  to,  C(',t)  is  smooth  and  convex. 


Theorem  4  means  that  just  as  in  the  Euclidean  case,  a  non-convex  curve  first  becomes 
convex  when  evolving  according  to  (21).  After  this,  the  curve  converges  to  an  ellipse  from 
our  results  in  [118].  Because  of  this,  and  other  related  properties  (see  [119]),  we  can  conclude 
that  equation  (21)  is  the  affine  analogue  of  (17)  for  smooth  embedded  curves,  and  thus  is 
called  the  affine  shortening  flow.  (It  is  also  the  affine  invariant  formulation  of  the  geometric 
heat  equation.)  In  the  next  section,  we  will  use  it  to  construct  an  affine  invariant  scale-space 
for  planar  shapes. 

4.2  Affine  invariant  scale-space 

In  the  previous  section,  we  indicated  that  equation  (21)  is  the  affine  analogue  of  (17).  Of 
course,  in  addition  to  the  Euclidean  invariant  property  of  the  Euclidean  curve  shortening  flow. 
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this  affine  flow  admits  also  unimodular  affine  invariant  solutions.  Due  to  this  analogy,  and 
certain  key  properties  [119],  we  can  use  the  affine  shortening  flow  for  defining  a  new  affine 
invariant  scale-space.  More  precisely,  the  representation  of  shape  obtained  from  the  evolving 
solution  of  equation  (21),  defines  the  affine  invariant  scale-space.  The  scale  parameter  is  given 
by  the  time  t 

Note  the  curve  shrinks  when  evolving  according  to  (21).  This  curve  can  be  normalized  in 
order  to  keep  constant  area.  In  fact,  in  [120],  we  have  evolutions  that  preserve  either  area 
or  length.  The  aflSne  scale-space  can  therefore  be  re-defined  by  means  of  these  normalized 
dilated  curves.  Since  the  process  of  normalization  modifies  k,  and  other  geometric  properties 
just  by  constant  multiplication  [118],  all  the  key  scale-space  features  for  the  family  of  curves 
C  satisfying  (21),  also  hold  for  the  normalized  scale-space. 

When  defining  scale-spaces,  a  number  of  desired  properties  must  be  verified  [75,  88,  110, 
138].  One  of  the  most  important  properties  is  the  causality  criterion^  which  requires  that 
no  new  features  should  be  introduced  in  the  curve  (or  image)  in  passing  from  fine  to  coarse 
scales  in  the  scale-space.  In  general,  “features”  are  related  to  the  salient  characteristics  of  the 
signal,  which  are  important  for  the  image  description,  and  are  easily  identified. 

Hummel  in  [75]  showed  that  the  causality  criterion  may  be  expressed  in  terms  of  the 
Maximum  Principle  [112].  The  Maximum  Principle  states  that  under  certain  conditions, 
a  given  function  satisfying  a  parabolic  partial  diflferential  equation,  attains  its  maximum 
(minimum)  on  the  boundary  [112].  (The  Maximum  Principle  is  very  frequently  used  in  the 
theory  of  curve  evolution  for  proving  important  properties.)  Hummel  showed  that  under 
certain  conditions,  when  the  Maximum  Principle  holds  for  a  given  parabolic  operator,  which 
defines  the  scale-space,  zero-crossings  are  never  created  at  a  non-zero  scale.  (This  result  was 
used  in  [110]  for  demonstrating  the  causality  principle.)  In  our  work  [5,  119],  we  show  that 
the  aflBne  shortening  flow  satisfies  a  Maximum  Principle,  and  so  because  of  this  and  other 
related  properties,  the  affine  invariant  scale-space  is  an  ideal  vehicle  for  the  multi-scale  study 
of  shape. 

4*3  Visual  Tracking 

Much  of  our  recent  research  in  image  processing  and  computer  vision  has  been  motivated  by 
problems  in  controlled  active  vision,  especially  visual  tracking.  We  have  already  described 
some  of  the  relevant  work  in  control  above,  and  so  we  would  like  to  consider  now  some  of  the 
key  tools  we  plan  to  employ  from  our  work  in  computer  vision  and  image  processing.  These 
include  active  contours,  optical  flow  and  stereo  disparity,  and  certain  results  from  invariant 
theory  for  invariant  object  recognition,  as  well  as  the  curve  and  surface  evolution  methodology 
sketched  above  in  Section  4.  These  methods  have  played  an  integral  part  in  our  study  of  the 
utilization  of  visual  information  in  a  feedback  loop. 

4.4  Geometric  Active  Contours 

In  this  section,  we  will  describe  a  new  paradigm  for  snakes  or  active  contours  based  on  prin¬ 
ciples  from  geometric  optimization  theory.  Active  contours  may  be  regarded  as  autonomous 
processes  which  employ  image  coherence  in  order  to  track  various  features  of  interest  over 
time.  Such  deformable  contours  have  the  ability  to  conform  to  various  object  shapes  and  mo¬ 
tions.  Snakes  have  been  utilized  for  segmentation,  edge  detection,  shape  modeling,  and  visual 
tracking.  Active  contours  have  also  been  widely  applied  for  various  applications  in  medical 
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imaging.  For  example,  snakes  have  been  employed  for  the  segmentation  of  myocardial  heart 
boundaries  as  a  prerequisite  from  which  such  vital  information  such  as  ejection-fraction  ratio, 
heart  output,  and  ventricular  volume  ratio  can  be  computed. 

In  the  classical  theory  of  snakes,  one  considers  energy  minimization  methods  where  con¬ 
trolled  continuity  splines  are  allowed  to  move  under  the  influence  of  external  image  dependent 
forces,  internal  forces,  and  certain  contraints  set  by  the  user.  As  is  well-known  there  may 
be  a  number  of  problems  associated  with  this  approach  such  as  initializations,  existence  of 
multiple  minima,  and  the  selection  of  the  elasticity  parameters.  Moreover,  natural  criteria 
for  the  splitting  and  merging  of  contours  (or  for  the  treatment  of  multiple  contours)  are  not 
readily  available  in  this  framework. 

In  [81],  we  have  proposed  a  novel  deformable  contour  model  to  successfully  solve  such 
problems,  and  which  will  become  one  of  our  key  techniques  for  tracking.  Our  method  is 
based  on  the  Euclidean  curve  shortening  evolution  (see  Section  4.1)  which  defines  the  gradient 
direction  in  which  a  given  curve  is  shrinking  as  fast  as  possible  relative  to  Euclidean  arc-length, 
and  on  the  theory  of  conformal  metrics.  Namely,  we  multiply  the  Euclidean  arc-length  by  a 
function  tailored  to  the  features  of  interest  which  we  want  to  extract,  and  then  we  compute  the 
corresponding  gradient  evolution  equations.  The  features  which  we  want  to  capture  therefore 
lie  at  the  bottom  of  a  potential  well  to  which  the  initial  contour  will  flow.  Further,  our  model 
may  be  easily  extended  to  extract  3D  contours  based  on  motion  by  mean  curvature  [81,  139]. 

Let  us  briefly  review  some  of  the  details  from  [81].  First  of  all,  in  [35]  and  [95],  a  snake 
model  based  on  the  level  set  formulation  of  the  Euclidean  curve  shortening  equation  is  pro¬ 
posed.  More  precisely,  the  model  is 

^  =  .^(x,!/)||V»||(div(]i||]])  +  -')-  (22) 

Here  the  function  y)  depends  on  the  given  image  and  is  used  as  a  “stopping  term.”  For 
example,  the  term  (l>{x,y)  may  chosen  to  be  small  near  an  edge,  and  so  acts  to  stop  the 
evolution  when  the  contour  gets  close  to  an  edge.  One  may  take  [35,  95] 


^  := 


1 

1  +  ||VG,*/||2’ 


(23) 


where  7  is  the  (grey-scale)  image  and  Go  is  a  Gaussian  (smoothing  filter)  filter.  The  function 
^{x,y,t)  evolves  in  (22)  according  to  the  associated  level  set  flow  for  planar  curve  evolution 
in  the  normal  direction  with  speed  a  function  of  curvature  which  was  introduced  in  [102,  123, 
124]. 

It  is  important  to  note  that  the  Euclidean  curve  shortening  part  of  this  evolution,  namely 


d\D  V® 


(24) 


is  derived  as  a  gradient  flow  for  shrinking  the  perimeter  as  quickly  as  possible.  As  is  explained 
in  [35],  the  constant  inflation  term  v  is  added  in  (22)  in  order  to  keep  the  evolution  moving 
in  the  proper  direction.  Note  that  we  are  taking  $  to  be  negative  in  the  interior  and  positive 
in  the  exterior  of  the  zero  level  set. 

We  would  like  to  modify  the  model  (22)  in  a  manner  suggested  by  Euclidean  curve  short¬ 
ening.  Namely,  we  will  change  the  ordinary  Euclidean  arc-length  function  along  a  curve 
C  =  {x{p).,y{p)y  with  parameter  p  given  by 


ds={xl  +  ylfl‘^dp, 
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to 


ds^  =  {xl  +  ylfl'^(t>dp, 


where  ^(x,  y)  is  a  positive  differentiable  function.  Then  we  want  to  compute  the  corresponding 
gradient  flow  for  shortening  length  relative  to  the  new  metric  ds^. 

Accordingly  set 

/•i  Br" 

:=  /  r-^udp. 


Let 


-  dc,,,dc, 


denote  the  unit  tangent.  Then  taking  the  first  variation  of  the  modified  length  function 
and  using  integration  by  parts  (see  [81]),  we  get  that 


=  -X 


dC  -  -- 

{^^4>KN-{V4>-N)M)ds 


which  means  that  the  direction  in  which  the  perimeter  is  shrinking  as  fast  as  possible  is 
given  by 


(25) 


This  is  precisely  the  gradient  flow  corresponding  to  the  miminization  of  the  length  functional 
L^.  The  level  set  version  of  this  is 


^  =  <>l|V®||div(ij||ij)  +  V^.V». 


(26) 


One  expects  that  this  evolution  should  attract  the  contour  very  quickly  to  the  feature  which 
lies  at  the  bottom  of  the  potential  well  described  by  the  gradient  flow  (26) .  As  in  [35,  95] ,  we 
may  also  add  a  constant  inflation  term,  and  so  derive  a  modified  model  of  (22)  given  by 


<^||V’^||(div( 


V® 

ilwii 


)  +  i^)  +  V<^  •  V'^. 


(27) 


Notice  that  for  as  in  (23),  V<^  will  look  like  a  doublet  near  an  edge.  Of  course,  one  may 
choose  other  candidates  for  <j)  in  order  to  pick  out  other  features. 

We  now  have  very  fast  implementations  of  these  snake  algorithms  based  on  level  set 
methods  [102,  123].  Clearly,  the  ability  of  our  snakes  to  change  topology,  and  quickly  capture 
the  desired  features  will  make  them  an  indispensable  tool  for  our  visual  tracking  algorithms. 

We  are  also  studying  an  affine  invariant  snake  model  for  tracking  based  on  our  work  in 
[101].  (The  evolution  itself  works  using  a  level  set  model  of  as  discussed  in  Section  4.1.) 

Finally,  our  methods  are  extendable  to  3D  images.  Indeed,  we  have  developed  affine  in¬ 
variant  volumetric  smoothers  in  [100].  We  also  have  3D  active  contour  evolvers  for  image 
segmentation,  shape  modeling,  and  edge  detection  based  on  both  snakes  (inward  deforma¬ 
tions)  and  bubbles  (outward  deformations)  in  our  work  [81,  139].  We  also  intend  to  use  our 
affine  smoothers  in  movies  as  a  preprocessing  tool  for  motion  estimation. 
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